Accurate Information vs. Looks Good: Scientific vs. Preferred Rendering

Scientists use digital camera data as the input to their analysis of image processing algorithms. In this paper we measured the "engineered color errors" introduced by digital camera color processing. Camera manufactures build their color management systems using the sRGB design standard. Although that sRGB is a guideline in the beginning, the final firmware shows certain liberties taken to make the best preferred rendering of scenes. The ensemble of algorithms that perform the color balance, color enhancement, tone scale, and post-LUT for display and printing, create large discrepancies between the sRGB measurements of the light from the scene, and actual sRGB values in cameras. We measured these discrepancies. These modifications to scene information introduce large changes in spatial information and make computer vision algorithms less accurate. Camera firmware and software modify color separation data for better looking pictures. These modifications need to be removed for accurate scientific scene analysis. We describe a computer program that converts a RAW digital camera file to calibrated file, in which digit value is proportional to log scene radiances. Introduction Image capture is the first step in computer vision. Spatial information processing is key to many recognition problems. Face, object, motion, feature recognition, and high-dynamic range imaging begin with spatial scene information. It would be nice to assume that, when we open a digital camera file and read the RGB digital values of a pixel, we get calibrated radiance measurements of the light from the scene. Cameras are designed to make attractive pictures using many firmware algorithms. Optical limits and firmware image processing modify scene radiance information, thus altering the scene's spatial relationships in the captured image. • High-end professional digital cameras modify the output Jpeg data to reduce the effects of optical vignetting and other optical artifacts. This approach uses specific camera and lens metadata to reduce known artifacts. • The light on the camera image plane is scene radiance plus scene-dependent veiling glare [1]. Although veiling glare removal would be highly desirable, it is not possible [2]. • Spectral information is not accurately recorded in standard digital color separation data. The improvements in color photography from the use of color masking are found in most color film reproduction processes. The technique was introduce by Albert [3] in 1889. Calibration of scene radiance vs. camera response is required for scientific use of color digital camera output. If a proper calibration is done, is it possible to eliminate "engineered errors"? The primary question here is the size of the discrepancies between the scene's actual radiance measurements, and the digital rendition stored in the memory of the camera. After all, that stored image is the data that we use to analyze and manipulate images. The achromatic function of sRGB is not significantly different from Munsell Value. As well, color cameras show only small deviations from reproducing Munsell Value. However, the chromatic errors are quite large, much greater than we expected. Camera responses stored in memory are highly enhanced chroma renditions. These transformations make good-compromise scene renditions for most photographic subjects. ! However, these nonlinear chroma enhancements seriously distort the results of multiple exposure techniques to capture HDR scene information. We report on the size and nature of these departures from the sRGB scene measurements. Although these departures from accurate scene rendition are well known to camera engineers [4-11, 27], there are many computer experiments in which the actual magnitude of these errors is larger than expected by their authors. These departures from accurate scene radiances can have significant adverse effects on image processing algorithms, such as: • Object recognition • Color balance from image statistics • Retinex HDR processing Figure 1 is a photograph of multiple objects with identical matte surfaces in nonuniform illumination. Sunlight is the only light source so that the spectral distribution of the illumination is constant. Figure 1. Jpeg photograph of oranges, lemons and limes in sun and